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Abstract. The majority game, modelling a system of heterogeneous agents 
trying to behave in a similar way, is introduced and studied using methods of 
statistical mechanics. The stationary states of the game are given by the (local) 
minima of a particular Hopfield like hamiltonian. On the basis of a replica 
symmetric calculations, we draw the phase diagram, which contains the analog of 
a retrieval phase. The number of metastablc states is estimated using the annealed 
approximation. The results are confronted with extensive numerical simulations. 
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1. Introduction 

The cooperative processes taking place in complex systems of agents interacting with 
each other, which were up to quite recently studied mainly by human sciences, became 
also an interesting object of research for physicists. Despite the tremendous complexity 
of such systems and the presence of the unpredictable ingredients related to human free 
will; some of the statistical regularities which characterise their collective behaviour 
can be studied using models and techniques developed in the field of statistical physics. 

As usually in such situations one builds very abstract theoretical models which 
have a much wider applicability. The so called minority game, for example, was 
proposed as a formalisation of the El-Farol bar problem in order to capture the 
essential features of traders' interaction in a stock exchange market But the 
same model can describe a more general situation where many agents compete for the 
exploitation of a number of scarce resources 

This seemingly simple model turns out to exhibit a surprisingly rich variety of 
complex behaviours which were studied thoroughly using various methods 0] O E] . 
The statistical mechanics approach to this sort of models is particularly interesting as 
it reveals aspects of cooperative phenomena which may be qualitatively different from 
those studied in physics. 

The minority game is based on the assumption that agents prefer to avoid crowds. 
Hence they tend to use those strategics which let them be as often as possible in the 
minority. This is only one of many possible types of interactions. It seems natural to 
ask what happens under the reversed (majority) rule, favouring those strategics which 
allow the agents to stay on the side of the majority. 
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In the economic interpretation of this sort of models, it has been shown [7] 
that the minority rule describes so-called "fundamental" or "contrarian" traders in 
a financial market. These are agents who believe that market prices are close to an 
equilibrium and hence expect that price fluctuations tend to generate price changes 
towards the equilibrium value. The majority rule is instead appropriate for trend 
followers, whose behaviour is thought to be responsible for the so called "bubbles" - 
buy rushes leading to price increases well beyond those which would be justified by 
an economic evaluation. The majority mechanism is self-reinforcing as it generates 
self-fulfilling prophecies: if the agents expect that the price will rise (fall), the majority 
of them will buy (sell) which will actually make the price rise (fall). 

Before focusing on the majority game, let us mention that the competition 
between majority and minority players has been studied in Ref. [Jj in the simplest 
setting and in Ref. |H] in its full complexity. 

From a wider perspective, the majority game describes a situation where the profit 
of agents increases with the number of agents acting in the same way. Conformity 
effects of this type are evident in the spreading of fashions. A further example may be 
that of a shop which lowers the prices as the number of customers increases, or of a 
product which becomes cheaper the more it is popular. This mechanism, which goes 
under the name of increasing returns in economics, lies at the heart of quite interesting 
aggregation phenomena - for example the emergence of cities and economic districts 
such as Silicon Valley and Hollywood 

As we shall see the study of majority game leads to the analysis of models which 
are very similar to attractor neural networks, in particular to the Hopfield model 
In brief, agents' learning dynamics is different from Glauber dynamics, but the 
energy landscape where it takes place is the same. It turns out that aggregation in the 
majority game is the same phenomenon as memory retrieval in the Hopfield model. 
Hence the physics of neural networks tells us a lot about the behaviour of the majority 
game. On the other hand, this study also provides new results on the physics of neural 
networks by probing the energy landscapes such as that of the Hopfield model with a 
different type of dynamics. 

Our work is based on a statistical mechanics approach to the stationary states of 
the majority game and its results are verified by numerical simulations. The paper 
is organised as follows: In the next section we introduce the model and in section 3 
we discuss its stationary states ~ we show that they can be identified with the local 
minima of the Hopfield type hamiltonian. Section 4 deals with the calculation of free 
energy and the construction of the phase diagram. Given that stationary states are 
selected in a dynamical way and not according to a Boltzmann weight, we compare 
our results with extensive numerical simulations. In order to clarify the dynamical 
behaviour of the model, we discuss the number of stationary states in section 5. We 
conclude with a summary of the main results and a discussion of their implications. 

2. The definition of the model 

We consider a system consisting of N agents interacting at discrete time intervals (at 
each round of the game). The interaction takes place through the action, concerning p 
objects or resources, which each agent undertakes. The actions are determined by one 
of r strategies which are chosen randomly and independently for each of the agents 
at the beginning of the game. In the course of the game the agents can change their 
actions only by changing their strategies, i.e. choosing one of the r predefined ones 
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(for each agent). We denote by a^^, the action taken by agent i ~ 1, . . . ,N concerning 
resource fi — 1, . . . ,p when he/she adopts strategy s = 1, . . . , r. We consider here the 
case of binary actions af^ = ±1. The specific vahies of a''^ - the reahsation of quenched 
disorder - are drawn at random from some distribution. Thus a strategy is a binary 
vector, which can be interpreted for instance as a hst of actions to be undertaken 
concerning each of the objects. 

Agent i chooses the strategy used in the next round of the game on the basis of 
the performances of his/her strategies in the previous runs, which are measured by a 
score functions Uis {t) . The agents choose the strategy 

Si{t) = argmax-Uis(i) (1) 

with the highest score and undertake the profile of actions af^.^j-j 1^ = ^,---,P {i = 
l,...,iV). The agents know about the actions of others only through the cumulative 
actions: 

N 

^"W = E<(*)' M=1,...,P (2) 

1=1 

which are used to update the score functions: 



p 

e X - 



{t + l) = u.,S) + -^a>t, M\t)-i^{a^ ,^,~at,) , (3) 



where e > and 77 e [0, 1] are constants. 

Let us first discuss this dynamics for 77 = 0. Note that A^^{t) has the same sign as 
the action undertaken by the majority concerning object /x. Then eq. (O implies that 
those strategies prescribing an action aligned with the majority are rewarded. In other 
words, by this learning dynamics, agents strive to find that strategy which puts them 
in the majority. Because of the averaging over ^ this rule is called a batch version 
of the majority game. The on-line version, where a value is randomly drawn at 
each time and agents update the scores depending on ^^^*^(t). will be discussed in the 
concluding section. 

With parameter < 77 < 1 one can change the degree to which the agents take 
into account the influence of their own actions on the cumulative quantity A^^it) (see 
In particular the case 7; = 1 describes agents who are learning to respond 
optimally to the behaviour of the others. Indeed for 77 = 1 eq. ||2Jl computes the 
correct value of A^^{t) if agent i had actually played strategy s. This is what game 
theory assumes a rational player should do, so the stationary states of the game for 
77 = 1 are Nash equilibria (i.e. those states where each agent takes the optimal strategy, 
given the strategy of others ^2]). Like in the Minority game in spite of the fact 
that A^{t) ~ ^/N is much larger than af^ ~ 0(1), the rj term is not negligible. 

The new, updated payoff functions are used to determine the action in the next 
time step through eq. iQJ. 

Note that A^^{t) is the difference between the size of the two groups of agents 
who undertake opposite actions afj,. = +1 or — 1. If agents do not interact and the 
actions +1 and —1 are equivalent, it is obvious that A^(i) ^ \/N . We shall pay special 
attention, in what follows, to the possibility that, when turning on the interaction, a 
macroscopic difference A'^it) ex. N may emerge, for some value of /i. 
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2.1. r ~ 2 case 

In this paper we will focus on the case where r = 2. We allow for a correlation of the 
two strategies of the same agent by introducing a parameter 

5 = P(ar+=<_) , (4) 

withP(af+ = l) = P(af+ = -l) = i. 

Instead of keeping track of the two payoff functions it is enough to consider their 
difference: 

y^ = (5) 



3. Stationary states 

Taking the average over the stationary state distribution, which we denote by (...), of 
the dynamical equation of we get: 

V. ^ ^y^^'+'^-y'^^'^^ ^m+f: - v^m. , (6) 

where we used a standard notation: 

MM M I M 1 ^ 

= (signy,), = , = 2^ — , X = - 2^ 

We expect that all the agents in the long time limit are frozen, i.e. they do not 
change their strategies. Indeed, exactly as in the case of the minority game, it is easy 
to check that the stationary states correspond to the minima of 



i,j i i 

The argument starts by observing that if Vi ^ 0, then ±00, depending on the 

sign of Vi, and hence = signw,;. Only if = we can have ^ ±1. Then one 
observes that zu — — §^ so these conditions are equivalent to the conditions for the 
minima of 7i^. But it is evident, from the form of 7i^, that its minima lie only at the 
corners of the hypercube [—1,1]^. 

The conclusion that stationary states correspond to the minima of 7i,, is 
straightforward from the dynamical equations in the limit e — > 0. Then one can 
introduce a rcscalcd continuous time t ~ et and verify that 7i,, is a Lyapunov function 
of the continuum time dynamics. 

Fig. n shows that numerical simulations fully confirm the above picture. Note in 
particular that while the initial stages of the dynamics are somewhat noisy, fluctuations 
are negligible in the long time limit. 

Strictly speaking there is no stationary state in terms of the variables as they 
diverge to ±00. The term stationary state refers to the variables m,j which take well 
defined values in the limit t ^ 00. 

We remark that the Tio is simply related to the predictability H — — 27io 
introduced in the minority game 0. Since rui = ±1 for all i, H is also equal to 
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the volatility = A^. Hence agents in the majority game strive to maximise H 
whereas agents in the minority game minimise it. 

Tiri is very similar to the Hamiltonian of the Hopfield model known from the 
theory of neural networks ^31 • The only differences are: the scaling with 1 /p instead 
of 1/N, the presence of the random field and the fact that the patterns can take 
three values 0, ±1 instead of only two ±1. By analogy with physics we shall call 7i,, 
energy. We notice that for g = one obtains the "pure" Hopfield model (with different 
rescaling), whereas for g = 1/2 one has a majority game with independently chosen 
strategies. 
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Figure 1. Simulations of the model for a 
initial conditions. 



0.2, N = 1000, ri = and random 



The stationary state values of rui satisfy the equations: 



m.( = sgn \ 



N 



(8) 



It is clear that any configuration C ~ {rrii} which is a solution of these equations for 
some value oi rj G [0, 1] will also be a solution for all rj' < rj. Hence the set Sjj of 
stationary states satisfies the property iS^ C Srj' for rj' < rj and, in particular, Si C S,j 
for all 77 < 1. It is also easy to see that the state with minimal value of Hjj lies in Si 
for all r] e [0, 1]. This shows that Nash equilibria are stationary state of the majority 
game for all values of rj, but the converse is not true. 

In the remaining part of this article we will try to determine the nature of the 
stationary states of the majority game and calculate the number of such states. In the 
next section we will employ the replica method to analyse the thermodynamics (for 
T = 0) of the model defined by the Hamiltonian {Tj). Then, using the stability relation 
we will calculate the number of stationary states in the annealed approximation 
as a function of various parameters . 



4. Replica approach to thermodynamics 

In the light of the previous considerations, it is clear that stationary states of the 
majority game for all values of 77 € [0, 1] are determined by the minima of lying 
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at the corners of the hypercube [—1,1]^. Note that when we restrict ourselves to 
rrti ±1 the rj term in becomes an irrelevant constant. This is why all the results 
in this section do not depend on rj. 

As usual, we build a partition function corresponding to the Hamiltonian 0. 
introducing an inverse temperature /?. Using the replica method |14j . we perform the 
average over quenched disorder and obtain the free energy density function. Within the 
replica symmetric ansatz, the latter depends on the Edward Anderson order parameter 
q, the overlap b with pattern and the residual overlap r. In the limit (3 ^ oo we 
find 5 — > 1 with 

finite and the free energy takes the form: 



1 fe^ ct 1 far [ , , f,2//,„3^>, 

/ = + - ^ — n — r — -9 + (1 - -9)6" ^ 

la 2 a — (1 — g)x \ v 



The parameters 6, r and x should take the values optimising the free energy. We 
remark that the overlap h corresponds to the equilibrium value of jN . The capacity 
a. = vl^ ^ the reversed temperature /3 and g defined by are the parameters. 

The analysis of the solutions to the saddle point equations 

^ = ^ = ^=0 (10) 
dh ' dr ' dx ^ ' 

allows us to draw the phase diagram (see fig. [SJ- 

One can distinguish two phases: a spin glass phase (b = 0) and a retrieval phase, 
where besides a spin glass solution there exists also a retrieval solution (6 7^ 0). 

A simple calculation shows that for the spin glass solution 



N N y \l na J 

within the replica symmetric ansatz (see Fig. The solution with 6 7^ 0, which 
using the neural network nomenclature, we call retrieval, is conveniently described in 
terms of the parameter x = b/ {2ay/ar) J^j, which satisfies the equation 

(l-g)erf(a;) 

X ~ — — . (11) 

V2a(l - 5) + 27r-l/2(l _ g)[g + (1 _ g)e-^^] 

For g < 2/3, two non-zero solutions for x{a) exist up to a critical value ac{g), but 
only one of them represents a thermodynamically stable state. Of course for g ^ 
we find that the spin glass and retrieval solutions of (|10() converge to the solutions 
found for the Hopfield model ^1]. The phase separation line adg) in fig. JSJ smoothly 
approaches the a = axis ac — |f (| — g) when g ^ 2/3. 

In the retrieval phase the dynamics can, depending on the initial conditions, end 
up in one of the two qualitatively different sorts of attractors. In the majority game 
language the retrieval corresponds to the macroscopic value of ^ 0{N) for some 
H (say 1), whereas ~ for the remaining values of fi ~ 2, ...,p. In the spin glass 
phase, A^^ ~ ViV for all /x. 
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Figure 2. The phase diagram computed by the replica symmetric approach 
described in the text (full line). The points are numerical estimate of the phase 
boundary ac{g) obtained from the crossing of the maximal overlap for different 
system sizes, as shown in the inset (g = 0.15, rj = I, p = 64, 128 and 256, maximal 
overlap initial conditions). 



To confirm these analytical results we have performed extensive numerical 
simulations using the dynamical definition of the model |^ This is important 
because the above calculation is based on the Boltzmann weight and it focuses on 
the lowest energy minima. There is, however, no guarantee that the dynamics of the 
majority game selects the minima with the lowest energy. 

Direct iteration of the dynamics is slow because in the late stages the time interval 
between individual spin flips Si — > —Si becomes very large. A much more efficient 
algorithm is possible in the continuum dynamics for e ^ because then one can 
integrate easily the dynamics between two consecutive spin flips. Clearly when i 3> 1 
the continuum time dynamics coincides with the discrete time batch dynamics (e = 1). 
Indeed no noticeable difference between simulations with e = 1 and e — > was found 
in the typical properties of stationary states. 

The comparison of the energies obtained using different methods for few values 
of a in the spin glass phase is presented in fig. |3| 

The estimate of the phase separation line coming from the simulations is plotted 
in fig. [3 Each point was obtained from the crossing of the curves A^/N vs a for 
different system sizes, as shown in the inset. There is a good agreement with the 
analytical results. The static results are rj independent, but simulations clearly show 
that this parameter plays an important role in the dynamics. 

Fig. 0] shows that retrieval states are indeed attractors in the retrieval phase. 
Even when starting from initial conditions which have only a partial overlap with 
pattern /i = 1, the dynamics converges to the retrieval state both for 77 = and 1. 
Actually, retrieval is enhanced when rj = 0. 

The inset of fig. 21 shows how the overlap A^/N depends on a, in simulations 
with maximal initial overlap A^{t = 0)/N. While for large a the overlap essentially 
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Figure 3. Comparison of energy profiles obtained for g = 0.5 and rj = 0. 
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Figure 4. Overlap /N in the stationary state as a function of initial overlap 
at t = in the retrieval phase [a = 0.05 and g = 0.1) for r; = (♦ and A) and 1 
(O and □). Two system sizes (p = 64 and 128) are shown in order to appreciate 
finite size effects. Inset: Overlap as a function of a for 77 = (O , * and <C>) and 
77 = 1 {+ and x) for maximal initial overlap. The stable (solid line) and unstable 
(dashed) solutions of saddle point equations are also shown. 



vanishes (in the limit N 00) if 77 = 1, /N attains a relatively large value for 
7y = 0. The error bars for 77 = indicate that the distribution of the overlap is very 
broad for intermediate values of a; the overlap in a particular run can converge to any 
value in the interval [0, 1 — 5]. 

Fig. El shows that a very different scenario takes place in the spin glass phase. 
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While for 7] ~ 1 even starting from maximal overlap the dynamics converges to a spin 
glass state {{A^)/N ^ as l/VlV when N oo), for 77 = the stationary state 
preserves the initial overlap. In order to understand this behaviour it is useful to 
compute the number of stationary states of the majority game as a function of the 
parameters a, g and 77. 
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Figure 5. Same as fig. |l]but in the spin glass phase (« = 1 and g = 0.1). 



5. Number of stationary states 

In order to learn more about the phase space of the majority game we calculate the 
number of stationary states as a function of various parameters. The stability relation 
(|HJ) can be written in the form: 



mS,tA - riQ > 0, mi = ±1 
The average number of stationary states can be defined as: 



(12) 



(13) 



\ \{m,=±l} i=l 

where ((...)) stands for the average over random patterns (j^ . This quantity is not self 
averaging and diverges exponentially with A'^. Therefore we will calculate the annealed 
entropy: 



—Sa = — ln$ 
N N 



(14) 



Since the average over the disorder is inside the logarithm (annealed 
approximation) the calculations are straightforward (see ^1 E|) and lead to the 
following result: 



max |sa(c,c,r,f,7,7)[ 
r r r -v ^ J 



c,c,r,r,7,7 



(15) 
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Saic, c, r, r, 7, 7) = cc - a77 + a^rr - - In [2r + (7 - 1)2] 



In 



cosh(c) --(l-g) 



c '^crf 



2(i-.g)(^-7)-fc 



2./2(l - g)T 



+ e'^erf 



2(l-.9)('?-7) + |c 



2^/2(1 -5)r 



— g cosh(c)erf 



2(l-.g)(r7-7) 



2^/2(1 -ff)r 



(16) 



In fig. I^the annealed entropy is plotted as a function of 77 and 




Figure 6. 

V 



; dependence of entropy Sa for various values of the parameters g and 



For large values of a the number of stationary states increases dramatically when 
77 decreases from 1 to . As a grows Sa saturates at ln(2) for 77 7^ 1. This can be 
understood observing that when 



< (1-7^)^2^(1-7/) (l-g) 



eq. ((SI) is satisfied for both m; = ±1. In words, when the effective field A-i on spin 
i due to the other spins mj is weak enough (i.e. its absolute value is smaller than 
(1 — g){l — 77)), rrii can take both values. If it happens for every i all the states of the 
system (expressed in terms of configuration {rrii}) are stationary and thus = ln(2). 
For large a the effective fields become small in absolute value. Indeed £,iA^i is well 
approximated by a Gaussian variable with zero mean and variance 2|7io|/p ^ 1/a. 
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Figure 7. a dependence of entropy Sa for 77 = 1 and various values of g. Inset: 
Maximal entropy s^"^ as a function of g for r) = I and o — > 00. 

The strong 77 dependence of Sq displayed in the first plot in fig. |S1 explains the 
difference in the dynamical behaviours of the model in the spin glass phase with 77 = 
and ?7 = 1 (fig. EJ. Since for 77 = the stationary states are very dense in the phase 
space, the system does not move far from the initial state before it is trapped in one 
of the fixed points of the dynamics. Thus the initial overlap changes very little. In 
the case ?/ = 1 the number of stationary states is much smaller and the system goes 
far away from the initial state (the initial non-zero overlap vanishes for TV 00). It 
is important to remark that the states with a non-zero overlap which are stable for 
?7 = in the spin glass phase arc not attractors in the usual sense because their basin 
of attraction vanishes in the thermodynamic limit. 

It is easy to find Sa as a function of Energy E: 

Sa{E)^ max \ Sa{c,c,r,t,-f,j) + uE + ^--cA (17) 
c,c,r,f ,7,7,« L Z a ) 

or overlap b. 

Sa{b) = max | Sa(c, c, T, f , 7, 7) + u6 - uc i (18) 

The results are presented in figs. |Hland|51 

The energy dependence of Sa enables us to determine the average energy of the 
infinite system (see the curve on fig. O and explains the size of the error bars of 
the simulation results. The larger error bars are due to the wider distribution of the 
energy (smaller a). 

The b dependence of Sa deep in the retrieval phase has a different character 
than in the spin glass phase (see fig. IHJl. The gap (region where Sa{b) < 0) in the 
distribution in fig. |51 disappears close to the phase boundary. We suppose that in 
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Figure 8. Entropy Sa{E) as a function of energy for g = 0.5, r; = and various 
values of a 
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reality the gap vanishes precisely at this boundary (compare |17p. Unfortunately due 
to the inadequacy of the annealed approximation we are not in a position to draw 
more quantitative conclusions. 

This inadequacy of the annealed approximation used to calculate the entropy Sa 
is much more evident than in the case of the minority game with 77 = 1 |1G| . One 
can see it by comparing the maximal allowed value of the overlap bmax = 1 ^ ,9 with 
the maximal value of b for which Sq(&) > (fig. (S)). For all 5 > the function 
Saib) suggests the existence of stationary states with b > bmax- Also the discrepancy 
between the energy profiles (fig. ^ should be attributed to the use of the annealed 
approximation, which indeed overestimates the real quantities. 

We tried to calculate the quenched entropy using replica method (see QHDi but 
up to now we were not able to solve the arising numerical problems. 
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6. Conclusions and discussion 

We have shown that the stationary states of the majority game correspond to the local 
minima of a Hopfield type hamiltonian and are attained when all agents "freeze" , i.e. 
use always the same strategy. Stationary states are not necessarily Nash equilibria 
except when agents correctly account for their impact on the aggregate (77 = 1). 
Depending on the parameters, the system can be in one of two phases: a retrieval 
phase characterised by attractors with a macroscopic overlap ~ 0{N) and a spin 
glass phase with no retrieval. A macroscopic overlap can also be sustained, in the 
spin glass phase, for rj small. We attribute this phenomenon to the self-reinforcing 
term (1 — r])^fsi in the dynamics which causes a dramatic increase in the number of 
stationary states as rj decreases. These results extend to the on-line version of the 
game. Indeed the equations for the stationary states are the same and, since agents 
freeze in the long run, fluctuations play no role (contrary to the case of the minority 
game 

These results allow us to draw some suggestions on the behaviour of systems of 
interacting agents driven by conformity or by increasing returns. The occurrence of 
a macroscopic overlap ~ 0{N) may correspond to crowd effects such as fashions 
and trends, when a large fraction of agents behave similarly in some respect, or to 
economic concentration, when, for example, one particular place is arbitrarily selected 
for large scale investments. The development of these crowd effects requires: i) that 
the number of agents is large compared to the number of resources {a small), ii) a, 
sufficient differentiation between strategies of agents {g < 2/3) and Hi) a large enough 
initial bias (i.e. an initial macroscopic overlap) towards a particular resource, fashion 
or place. Finally crowd effects can be sustained under more general conditions (i.e. 
in the spin glass phase) if agents do not behave strategically, i.e. if they neglect their 
impact on the aggregate {ij small). 

Besides the relevance of the model as a system of heterogeneous interacting agents, 
it is also interesting as an example of non-Glauber dynamics in the energy landscape 
of Hopfield type hamiltonians. It is remarkable that, in spite of the fact that the 
dynamics of yi does not satisfy detailed balance, the statistical mechanics picture 
remains quite accurate. 
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